Skip to content

core: p2p message counters, tx-pool gauges and embedded-timestamp diagnostics - #879

Merged
frstrtr merged 1 commit into
masterfrom
core/p2p-message-counters
Jul 27, 2026
Merged

core: p2p message counters, tx-pool gauges and embedded-timestamp diagnostics#879
frstrtr merged 1 commit into
masterfrom
core/p2p-message-counters

Conversation

@frstrtr

@frstrtr frstrtr commented Jul 26, 2026

Copy link
Copy Markdown
Owner

Why

Right now we cannot answer "are all the p2p message types actually working on the wire?" — because not one of them emits a log line at any verbosity, and m_known_txs.size() is exposed nowhere: no log, no dashboard field, no JSON.

That blind spot has already cost us. A live production investigation had to resort to guesswork, and a log-grep returning zero was read as "this message type is not implemented" for code that was in fact fully wired and running. A grep of zero is not evidence when nothing ever logs.

This PR closes the blind spot. Observability only.

What

1. Per-message-type wire counters

New src/core/p2p_message_stats.hpp — a process-wide struct of relaxed atomics with an in/out pair for each canonical message type: version, verack, ping, pong, addrme, addrs, getaddrs, shares, sharereq, sharereply, have_tx, losing_tx, remember_tx, forget_tx, bestblock, plus an unknown catch-all bucket.

Incremented at the two shared choke points every pool message must pass through:

direction site file
inbound pool::NodeBridge::handle() src/pool/node.hpp
outbound pool::Peer::write() src/pool/peer.hpp

Both are shared code, so every coin lane is instrumented without touching a single per-coin protocol file (protocol_legacy.cpp / protocol_actual.cpp are untouched), and the numbers mean the same thing across coins.

verack and pong belong to the coin-daemon p2p layer, not the pool protocol. They are carried anyway so the endpoint answers the full operator-facing list rather than silently omitting two names; they read 0 on a healthy pool node, and that zero is an answer, not a gap.

2. Tx-pool visibility

known_txs_size, known_txs_order_size, last_have_tx_advert_size, last_losing_tx_advert_size, have_tx_adverts_sent.

This is what settles whether TXPOOL=0 on a peer dashboard means "our pool is genuinely empty" (known_txs_size == 0) or "the advert is being suppressed" (known_txs_size > 0, no adverts sent). known_txs_order_size renders as null on lanes with no recency deque (DASH) — deliberately distinct from 0.

Published by DASH from the 10 s advert sweep and from publish_snapshot(); by btc/ltc/dgb from prune_shares().

3. Sharechain embedded-timestamp diagnostics (DASH)

  • tip_lag_seconds — wall-clock now minus the chain tip's embedded share_data.timestamp
  • saturation_fraction — over the last 100 shares, the proportion of embedded deltas equal to the clip upper bound 2*SHARE_PERIOD-1 (39 s on DASH), alongside the raw delta_samples / delta_saturated / clip_upper_bound

Upstream p2pool clips embedded share timestamps to [prev+1, prev+2*SHARE_PERIOD-1] (data.py:239-242). When real cadence exceeds that bound the embedded clock saturates: the difficulty retarget reads embedded timestamps, so it goes blind to hashrate, and the floor decays ~1.18 %/share. A 6.81 hour embedded lag was measured live.

These two fields are the honest early-warning signal. pool_hash_rate and min_difficulty are not — under saturation they are computed from that same saturated embedded history, so they measure floor decay rather than the pool. That is precisely why they are unfit as deploy criteria and these raw fields exist.

Computed in publish_timestamp_diagnostics(), called from DASH's publish_snapshot() — already under the exclusive tracker lock at every call site, bounded at 101 chain lookups per think cycle, no allocation beyond one 101-entry vector.

4. Endpoint

GET /p2p_stats — read-only. It calls nothing on the node and takes no lock: it loads relaxed atomics and formats them, so it cannot block the IO or compute thread. Routed in http_session.cpp; documented in docs/DASHBOARD_INTEGRATION.md.

{
  "messages": { "shares": {"in": 412, "out": 388} },
  "totals": { "in": 1204, "out": 1189 },
  "trace_enabled": false,
  "txpool": { "known_txs_size": 12043, "known_txs_order_size": null,
              "last_have_tx_advert_size": 500, "last_losing_tx_advert_size": 0,
              "have_tx_adverts_sent": 37, "updated_at": 1785049468 },
  "sharechain_timestamps": { "tip_embedded_timestamp": 1785024952, "tip_lag_seconds": 24516,
                             "clip_upper_bound": 39, "delta_samples": 100,
                             "delta_saturated": 97, "saturation_fraction": 0.97,
                             "updated_at": 1785049468 }
}

Safety

  • Zero consensus impact. Counters and read-only display fields. Nothing here is read by share validation, minting, payout, coinbase construction or peer behaviour; no share byte, wire byte or peer decision changes.
  • Hot path. One relaxed fetch_add per message plus a length-first string_view compare against at most 15 short literals.
  • No per-message logging by default. A debug trace exists behind trace_enabled, defaults to false, and nothing in-tree switches it on.
  • Command strings are NUL-trimmed before matching — the inbound counter runs before MessageHandler::parse() strips the padding, so without that trim every inbound message would land in the unknown bucket. Covered by a KAT.

Tests

Folded into the existing allowlisted core_test target (src/core/test/CMakeLists.txt) — never a new add_executable, which is not in build.yml's --target list and so silently reports "Not Run".

12 new cases in src/core/test/p2p_message_stats_test.cpp: the full command-to-type mapping, the NUL-padded wire form, unknown/prefix/superstring/case rejection, per-direction counter isolation, trace-off-by-default, and the saturation detector (fully saturated, healthy, mixed fraction, off-by-one exactness, degenerate input, out-of-order pairs).

[==========] 101 tests from 16 test suites ran. (5 ms total)
[  PASSED  ] 101 tests.

Regression sweep, all green: test_dash_node (8), test_dash_poolnode_messages (10), test_dash_node_reception_wire (22), test_dash_peer (5), test_dash_share_tracker (9), test_dash_p2p_node (23). Binaries c2pool, c2pool-btc, c2pool-dash build clean.

Live wire proof

Two c2pool-dash instances on loopback, one dialling the other:

$ curl -s http://127.0.0.1:18102/p2p_stats
{'version': {'in': 1, 'out': 1}, 'addrs': {'in': 1, 'out': 1},
 'getaddrs': {'in': 1, 'out': 1}, 'have_tx': {'in': 1, 'out': 1}}
totals {'in': 4, 'out': 4}

Both directions counted on a real handshake, including the have_tx advert. A single-node run confirmed clip_upper_bound: 39 and known_txs_order_size: null on the DASH lane, HTTP 200.

Not done / flagged

  • .github/workflows/ is not modified (token lacks workflow scope). None was needed: core_test is already on both the Build-tests and the sanitizer --target lists.
  • The timestamp diagnostics are wired for DASH only so far. The header is coin-agnostic; the other lanes need the same one-line publish_timestamp_diagnostics() call in their snapshot path to light up.

Shared-base change — isolation-rule exception (on the record)

Per the per-coin isolation rule, this PR is flagged as a shared-base change and justified as such: the only edits outside src/core / src/pool are one-line instrumentation calls at two shared choke points — pool::NodeBridge::handle() (inbound) and pool::Peer::write() (outbound) — reached in src/impl/{btc,dgb,ltc}/node.cpp and src/impl/dash/node.hpp. No per-coin protocol file is touched (protocol_legacy.cpp / protocol_actual.cpp untouched), the counters mean the same thing on every lane, and all coin smokes are green. Observability only; no consensus/payout/wire-format behaviour changes.

The DASH-only publish_timestamp_diagnostics() gap (above) is deliberately left as a follow-up — the other lanes each need the same single-line call in their snapshot path — so this PR stays narrow and can land now.

Rebased 2026-07-26

Rebased onto origin/master (e56fe92a; picked up #870/#868/#867 and #863). One conflict, src/core/test/CMakeLists.txt: master added socket_write_queue_test.cpp (#863) to the shared core_test add_executable; this PR adds p2p_message_stats_test.cpp. Resolved as a union — both comment blocks kept, both sources in the single allowlisted add_executable (no new target, no #769 "Not Run" trap). GPG-signed, no attribution trailers.

@frstrtr
frstrtr force-pushed the core/p2p-message-counters branch from 1c6acd2 to 6a5efe6 Compare July 26, 2026 07:39
@frstrtr
frstrtr marked this pull request as ready for review July 26, 2026 11:05
@frstrtr
frstrtr force-pushed the core/p2p-message-counters branch from 6a5efe6 to 9c5ee3e Compare July 26, 2026 11:29
…gnostics

Not one of the canonical p2pool pool-protocol message types emitted anything
at any verbosity, and m_known_txs.size() was exposed nowhere -- no log, no
dashboard field, no JSON. A live investigation had to guess, and a log-grep
returning zero was misread as "not implemented" for code that was fully wired
and running. A grep of zero is not evidence when nothing ever logs.

Adds core/p2p_message_stats.hpp: a process-wide struct of relaxed atomics,
incremented at the two choke points every pool message must pass through --
pool::NodeBridge::handle() inbound and pool::Peer::write() outbound. Both are
shared code, so every coin lane is instrumented without touching a single
per-coin protocol file, and the numbers mean the same thing across coins.

Read back through a new read-only endpoint, GET /p2p_stats:

  messages.<type>.{in,out}   per-type wire counters for version, verack, ping,
                             pong, addrme, addrs, getaddrs, shares, sharereq,
                             sharereply, have_tx, losing_tx, remember_tx,
                             forget_tx, bestblock, plus an unknown bucket
  totals.{in,out}
  txpool.known_txs_size              m_known_txs.size()
  txpool.known_txs_order_size        m_known_txs_order.size(), null where the
                                     lane has no recency deque (DASH)
  txpool.last_have_tx_advert_size    size of the last have_tx put on the wire
  txpool.last_losing_tx_advert_size
  txpool.have_tx_adverts_sent
  sharechain_timestamps.tip_lag_seconds       now - tip share_data.timestamp
  sharechain_timestamps.saturation_fraction   share of the last 100 embedded
                                              deltas pinned to the clip bound
  sharechain_timestamps.clip_upper_bound      2*SHARE_PERIOD-1 (39 s on DASH)
  sharechain_timestamps.{delta_samples,delta_saturated,tip_embedded_timestamp}

The txpool block settles whether TXPOOL=0 on a peer dashboard means our pool
is genuinely empty or the advert is being suppressed.

The timestamp block exists because the aggregate gauges proved untrustworthy.
Upstream p2pool clips embedded share timestamps to
[prev+1, prev+2*SHARE_PERIOD-1] (data.py:239-242); once real cadence outruns
that bound the embedded clock saturates, the difficulty retarget goes blind to
hashrate and the floor decays. A 6.81 hour embedded lag was measured live.
pool_hash_rate and min_difficulty are computed from that same saturated
history, so they report floor decay rather than the pool -- tip lag and
saturation fraction are the honest early-warning pair.

DASH publishes the timestamp diagnostics from publish_snapshot() (already
under the exclusive tracker lock, bounded at 101 chain lookups per think
cycle) and the tx-pool gauges from the 10 s advert sweep; btc/ltc/dgb publish
their tx-pool gauges from prune_shares().

Observability only: counters and read-only fields. No consensus, minting,
share bytes or peer behaviour changes. Per-message logging stays off by
default behind a flag nothing in-tree enables.

Tests folded into the EXISTING allowlisted core_test target (never a new
add_executable -- that is the "Not Run" trap): 12 cases over the command
mapping including the NUL-padded wire form, the per-direction counters, and
the saturation detector.
@frstrtr
frstrtr force-pushed the core/p2p-message-counters branch from 9c5ee3e to c0df013 Compare July 26, 2026 22:51
@frstrtr
frstrtr merged commit f068a23 into master Jul 27, 2026
41 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant